Author Description

Summary By MatterAI

🔄 What Changed

This PR adds support for the 'thinking' input parameter for Gemini models in both Google and Google Vertex AI providers. The implementation allows passing a 'thinking' configuration that enables the model to include its thought process in responses and set a token budget for this feature.

🔍 Impact of the Change

This enhancement enables users to leverage Gemini's thinking capabilities through the gateway, allowing for more transparent AI reasoning and decision-making processes. The feature is implemented by transforming the 'thinking' parameter into the appropriate configuration format required by Google's APIs.

📁 Total Files Changed

• 4 files modified with 25 additions and 0 deletions
• Modified files:
  • src/providers/google-vertex-ai/chatComplete.ts
  • src/providers/google-vertex-ai/transformGenerationConfig.ts
  • src/providers/google/chatComplete.ts
  • src/types/requestBody.ts

🧪 Test Added

No explicit tests were added in this PR. Manual testing should be performed to verify the functionality of the Gemini thinking parameter in both Google and Google Vertex AI providers.

🔒 Security Vulnerabilities

No security vulnerabilities were identified in the changes.

Motivation

To enable users to access Gemini's thinking capabilities through the gateway, providing more transparency in AI reasoning and decision-making processes.

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update
• Refactoring (no functional changes)

How Has This Been Tested?

• Unit Tests
• Integration Tests
• Manual Testing

Screenshots (if applicable)

N/A

Checklist

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

Related Issues

N/A

Quality Recommendations

1. Add type safety for the thinking parameter by using a more specific type than Record<string, any> for thinkingConfig

2. Add validation for the thinking.budget_tokens parameter to ensure it's a positive number

3. Add unit tests to verify the transformation of the thinking parameter

4. Add documentation comments explaining the thinking parameter functionality

Sequence Diagram

sequenceDiagram
    participant Client
    participant Gateway
    participant RequestBody as RequestBody Types
    participant GoogleProvider as Google Provider
    participant VertexAIProvider as Google Vertex AI Provider
    participant GoogleAPI as Google API
    participant VertexAI as Google Vertex AI

    Client->>Gateway: Request with thinking parameter
    Gateway->>RequestBody: Validate thinking parameter
    Note over RequestBody: Added thinking interface<br/>with type and budget_tokens
    
    alt Google Provider
        Gateway->>GoogleProvider: Forward request
        GoogleProvider->>GoogleProvider: transformGenerationConfig()
        Note over GoogleProvider: Transform thinking parameter<br/>include_thoughts: true<br/>thinking_budget: params.thinking.budget_tokens
        GoogleProvider->>GoogleAPI: API call with thinking_config
        GoogleAPI-->>GoogleProvider: Response with thinking details
        GoogleProvider-->>Gateway: Processed response
    else Google Vertex AI Provider
        Gateway->>VertexAIProvider: Forward request
        VertexAIProvider->>VertexAIProvider: transformGenerationConfig()
        Note over VertexAIProvider: Transform thinking parameter<br/>include_thoughts: true<br/>thinking_budget: params.thinking.budget_tokens
        VertexAIProvider->>VertexAI: API call with thinking_config
        VertexAI-->>VertexAIProvider: Response with thinking details
        VertexAIProvider-->>Gateway: Processed response
    end
    
    Gateway-->>Client: Final response